CMU computer program reads human mind

Carnegie Mellon University scientists are training a computer to decode the human mind.

The scientists explain today in the journal Science how their computer program can read a person's brain scan and figure out what noun he or she is thinking. Their goal is to perfect the program so that it can help autistic, schizophrenic or paralyzed people.

"Imagine somebody who is completely immobilized, a quadriplegic, who can't talk," said co-author Marcel Just, a neuroscientist and director of Carnegie Mellon's Center for Cognitive Brain Imaging. "They could think a thought and maybe the machine could figure out what it is."

The work expands on a separate study the same scientists published earlier this year showing that people's brains activate similarly when they think about the same word.

In the latest study, the scientists used functional magnetic resonance imaging (fMRI) to measure changes in blood flow in the brain as volunteers thought about 25 verbs, such as "see," "hear," "taste," "run" and "push."

When given a noun -- such as "celery" -- the computer program searches for it in a database containing 1 trillion words of text. Once the noun is found, the program searches for nearby verbs.

The program predicts what it thinks the brain scan would look like if a person were thinking of that noun by blending the brain scans from the 25 verbs, giving more weight to the verbs that occur most frequently with the noun in the text.

So for celery, such verbs as "eat" and "taste" would dominate the predicted brain scan, while "ride" wouldn't even appear.

When given a brain scan of a noun, the computer program was 72 percent accurate in picking the word it was associated with out of a list of 1,001 nouns.

"The bottom line is -- and this is what's really new here -- nobody had previously even tried to build a theory or computational model that would predict neural activity for ... arbitrary words," said co-author and computer scientist Tom Mitchell, who heads the School of Computer Science's Machine Learning Department.

Prior to this study, scientists could only figure out the word a person was thinking if they already had a brain scan of that word, said John-Dylan Haynes, a neuroscientist at the Bernstein Center for Computational Neuroscience in Berlin. He did not participate in the research.

"This very interesting study now shows a method that allows one to read out a large number of different thoughts from brain activity, even with only a few calibration measurements," Haynes wrote in an e-mail.

Mitchell and Just have been tweaking their computer model to interpret simple sentences. They've also done small studies that show the brain scans of a bilingual person are the same, regardless of which language they use to think of a noun.

"Possible applications could include brain-computer interfacing, neuromarketing and even possibly detection of concealed thoughts for criminal investigations," Haynes said.

Just said such applications could be feasible but are still several years away because the fMRI machine is large and expensive, several computers have to be united to handle the data and people have to remain very still for long periods.

"Plus, you can't make somebody think something consistently," Just said. "All they have to do is think of something else."